May 2012 

Mitigation of the LHC Inverse Problem 



NlCKI BORNHAUSER 1 AND MANUEL DREES 1 



1 Physikalisches Institut and Bethe Center for Theoretical Physics, Universitdt Bonn, Nufiallee 

12, D-53115 Bonn, Germany 



Abstract 

The LHC inverse problem refers to the difficulties in determining the parameters of an 
underlying theory from data (to be) taken by the LHC experiments: if they find signals 
of new physics, and an underlying theory is assumed, could its parameters be determined 
uniquely, or do different parameter choices give indistinguishable experimental signatures? 
This inverse problem was studied before for a supersymmetric Standard Model with 15 
free parameters. This earlier study found 283 indistinguishable pairs of parameter choices, 
called degenerate pairs, even if backgrounds are ignored. We can resolve all but 23 of 
those pairs by constructing a true x 2 distribution using mostly counting observables. The 
elimination of systematic errors would even allow separating the residual degeneracies. 
Taking the Standard Model background into account we still can resolve 237 of the 283 
"degenerate" pairs. This indicates that (some of) our observables should also be useful 
for the purpose of determining the values of SUSY parameters. 



1 Introduction 



The Large Hadron Collider (LHC) is successfully running and collecting data. It is hoped that 
in the near future signs of "new physics" will show up. Once a signal for physics beyond the 
Standard Model (SM) has been established, one would need to identify the underlying theory, 
and to determine its parameters. For a given theory in an ideal world a certain parameter choice 
would lead to a unique experimental signature. In this case the "inverse problem", of going 
from experimental observables to parameters of the underlying theory, would have a unique 
solution. However, it is quite possible that - even within a given theory - several different sets 
of parameters reproduce all observables that are available at a given time. Note that we are here 
not concerned about (small) regions of parameter space centered around the true solution; it 
is clear that in the presence of non-vanishing experimental errors, a finite region of parameter 
space will be allowed even if the inverse problem does have a unique solution. Rather, the 
concern is that quite different, not necessarily connected regions of parameter space cannot be 
disentangled using only (future) LHC data. 

This issue has been studied in most detail in the framework of the minimal supersymmetric 
extension of the SM, the MSSM [I]. Most studies focused on determining the masses of specific 
sparticles using features of kinematic distributions, including invariant mass "edges" [2] or 
"kinks" in more complicated observables PJ. This uses only kinematical information, and 
ideally allows to directly determine the masses of the involved superparticles. These can then be 
compared to predictions of specific models of supersymmetry breaking, or to fix the parameters 
of such models. The model dependence thus enters only at the last stage of the analysis. 

One disadvantage of these kinematical methods is that dynamical information, i.e. informa- 
tion on counting rates which determine products of cross sections and branching ratios, is not 
used at all. It has been realized quite early that (ratios of) numbers of events of specific types 
can be used to discriminate between variants of the MSSM [HE]. More recently, information on 
the total event rate after cuts has been shown to improve the performance of purely kinematical 
fits [6]. Even in that case this method has generally only been used in constrained versions of 
the MSSM, with a rather small number of free parameters. This allows to focus on the most 
prominent kinematical features, since only a small number of masses needs to be measured in 
order to determine all free parameters of the theory. 

Another disadvantage of parameter reconstruction based on kinematic edges or kinks is 
that it is not straightforward to automatize them. Generally human intervention is required to 
detect an edge. This method is therefore well suited to detailed analyses of benchmark points 
(and eventually of real data, it is hoped), but cannot easily be used for broad scans of parameter 
space. 

In order to overcome the last two disadvantages, Arkani-Hamed et. al. [7] attempted a 
"brute-force" approach to the parameter reconstruction issue in the context of a quite general 
version of the MSSM with 15 free parameters defined at the weak (or superparticle mass) 
scaleE] They randomly generated 43,026 sets of parameters, called "models" by them. For each 
parameter set they Monte Carlo generated the number of events corresponding to an integrated 
luminosity of 10 fb _1 at a center of mass energy of 14 TeV. They analyzed these events using a 

* A fully general MSSM has many more free parameters. However, most of those arc related to flavor mixing 
and/or CP violation, and are strongly constrained by low-energy observables and/or have little influence on 
collider phenomenology. 
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total of 1808 observables. Most of them are kinematical observables (based on invariant mass 
and transverse momentum distributional]); less than ten percent of the observables are counting 
observables (number of events with a certain property). Based on a statistical analysis, 283 
so-called degenerate model pairs were identified. These are pairs of parameter sets that could 
not be distinguished by their method of comparison with an estimated 95 % confidence level [TJ. 

Note that the analysis of ref. [7] used a very large number of observables, but was still 
unable to distinguish between even quite different spectra, even though the sparticle mass scale 
did not exceed 1 TeV. This can be interpreted as implying that the LHC experiments are in 
principle incapable of determining all MSSM parameters in a model-independent fashion. 

However, the analysis of ref. [7J has several weaknesses. To begin with, initial state radiation 
and the "underlying event" (thought to arise from interactions between the "spectator partons" 
not taking part in the primary hard interaction) are ignored. These features enlarge the event; 
since they do not depend much on the produced final state, they can be expected to reduce the 
observable differences between different parameter sets even more, i.e. to increase the number 
of degenerate pairs. 

Another criticism is that Arkani-Hamed et al. use a single "x 2 — like" quantity, called 
(ASab) 2 , to analyze the information of all 1808 observables; here A, B stand for two sets 
of parameters, and (ASab) 2 essentially sums the squared differences between the predicted 
observables, divided by the squared total error of these observables, and normalized to the 
number of observables included. An observable is included in the definition of (ASab) 2 only 
if the total error is smaller than both predictions or smaller than the difference between the 
predictions; this is meant to reduce the dilution of (ASab) 2 by observables that are so poorly 
measured that they cannot discriminate between parameter sets. 

Nevertheless the dilution of the statistical measure by observables with little discrimina- 
tory power remains an issue. In fact, Arkani-Hamed et al. found [7J that 46 out of the 283 
"degenerate" pairs had (at least) one observable differing by more than 5 estimated standard 
deviations, even if one only considers the subset of observables constructed from final states 
containing two charged leptons. They do not consider this to be significant, since when com- 
paring 2,600 parameter sets with themselves, but run with different seed of the Monte Carlo 
event generator, they found 611 cases where (at least) one di-lepton observable differed by 
more than five estimated standard deviations between the two runs. 

This example casts serious doubt on the estimate of the standard deviation of (ASab) 2 
used in ref. [7J. Recall that for a normal (Gaussian) distributed observable, the probability of 
two measurements to differ by more than five (true) standard deviations is about 5.7 ■ 10~ 7 . 
By our count Arkani-Hamed et al. include around 1,000 di-lepton observables. Even if they 
were all statistically independent, one would expect at most 5.7 ■ 10~ 7 x 1000 x 2600 ~ 1.5 
observables to differ at more than five standard deviations. Since the observables used in 
ref. [7J are actually highly correlated, as acknowledged by Arkani-Hamed et al., the number 
of statistically independent di-lepton observables is much smaller than 1,000, reducing the 
probability to observe true 5a fluctuations even more. The fact that 611 such fluctuations were 
observed thus indicates a problem with the estimate of the error and/or with the generation of 
the events. 

t Rather than looking for edges or kinks in these distributions, they bin them in ten or 20 bins, such that 
each bin contains the same number of events. The observables are then the boundaries of these bins. These 
observables can obviously be constructed automatically, without human intervention. 



2 



A possibly related issue is that the correlations between observables are not included in the 
definition of (ASab) 2 [Z]; this is why this quantity is a "x 2 — like" variable, rather than a true \ 2 ■ 
This means that the statistical interpretation of this observable is not a priori clear. Arkani- 
Hamed et al. address this problem by again comparing parameter sets to themselves, generated 
with different seeds of the random number generator. They found that in 5 % of all cases 
(ASab) 2 > 0.285 even for the same parameter set. They therefore defined (ASab) 2 > 0.285 
as equivalent to two sets of parameters being distinguishable at 95 % confidence level. This 
inference is not obvious to us, since the statistical properties of (ASab) 2 are a priori unknown, 
and the comparison of parameter sets with themselves yielded results that seem violently at 
odds with the usual interpretation in terms of standard deviations. 

Finally, the variable (ASab) 2 is constructed to resemble a \ 2 comparing two different mea- 
surements, both of which are assigned the statistical uncertainties expected for a data sample of 
10 fb _1 . However, the issue is whether, given that Nature chose parameter set A, a measurement 
can exclude the 'prediction made for parameter set B. This prediction should (in principle) have 
negligible statistical uncertainty. It is not clear to what extent this distorts the statistics, since 
the cut-off for (ASab) 2 that defines which pairs are deemed indistinguishable is determined 
by Monte Carlo experiments; however, conceptually there is a significant difference between 
comparing two experiments and comparing an experiment with a prediction. 

In this paper we re-analyze the degenerate pairs of ref . [7] ; for statistical tests we also employ 
a larger sample of pairs with slightly larger (ASab) 2 ■ We include initial-state radiation as well 
as the underlying event, and analyze events at the hadron level. We want to construct a true 
X 2 , in order to have an observable with well-defined statistical properties. Moreover, we focus 
on counting signatures. One reason for this is that it is relatively easy to define statistically 
independent counting rates, by simply defining mutually exclusive classes of events. In contrast, 
since all events of a given class contribute to various kinematical observables, these observables 
will be statistically correlated. A second reason is that there is a much larger literature on using 
kinematical quantities [2],|3] than on the use of counting rates [1H6] . Here we want to show that 
counting rates can play an important role in discriminating between sets of MSSM parameters; 
we view this as a first step towards a determination of the values of these parameters, including 
statistically meaningful errors. Altogether we employ (at most) 84 observables when comparing 
sets of parameters. Since most of these observables are counting rates, which should not depend 
sensitively on details of the detectors, we do not use a detector simulation (unlike ref. [7]); 
however, we use realistic efficiencies when counting b— jets and r— leptons. Moreover, unlike 
ref. [7j we also investigate the effect of SM backgrounds. 

The remainder of this article is organized as follows. In the next Section we describe the 
definition of the sets of parameters, as well as some technical details of our simulation. In Sec. 3 
we list the observables we use, and construct an overall \ 2 variable out of them. We perform 
some checks to show that this variable has the desired statistical properties. Section 4 contains 
the results of our numerical analysis, and we conclude in Sec. 5. Details of the definition of 
objects (jets, leptons) and of the cuts employed are given in the Appendices. 
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2 The Simulation 



In this Section we give some details of our simulation. We start with a description of how the 
sets of MSSM parameters were chosen in ref. [TJ, and then describe how we generate events. 

2.1 Parameter Sets 

Here we follow ref. [TJ. There the MSSM spectrum was parameterized directly at the super- 
particle mass scale, using the following 15 free parameters: three gaugino masses Mi (bino), 
M.2 (wino) and M3 (gluino); four independent slepton masses mg £ = mp, L , m e R = m ji R , m f L 
and mf R ; six independent squark masses rriq 1L = rrig 2L , m,- 3I , rn%i R = ^c H , m l R -. m d R = m s R 
and wig ; the Higgs(ino) mass parameter /x; and the ratio of vacuum expectation values tan/3. 
The masses of the first and second generation sfermions with given quantum numbers are taken 
equal; this automatically satisfies stringent constraints on flavor changing neutral currents in 
this sector pQ. Four additional parameters are fixed, namely the trilinear scalar couplings 
A t = A b = A T = 800 GeV and the pseudoscalar Higgs pole mass = 850 GeV. Out of 
the A— parameters, only A t is expected to have significant influence on LHC phenomenology 
in the context of the MSSM, but determining its parameters is expected to be quite difficult. 
Fixing its value therefore reduces the difficulty of the overall problem. Similarly, determining 
m,4 is likely quite difficult, unless it is so small that the heavy Higgs bosons can be produced 
abundantly. 

43,026 sets of parameters were then generated in [TJ, by randomly chosing values of the free 
parameters, with probability distributions that are flat within certain ranges. In particular, 
the parameters Mi, M2 and \x and the four slepton masses lie between 100 GeV and 1 TeV. 
The gluino mass and the six squark masses lie between 600 GeV and 1 TeVjD and tan (3 varies 
between 2 and 50. The relations between the parameters are further restricted by the condition 

K^on < rrCXo + 50 GeV < m ™or + 100 GeV , (2.1) 

with m™^p ton being the maximum slepton soft mass, being the maximum of Mi, M 2 and /i, 

and Jn™^ being the maximum soft mass or mass parameter of any color-charged superparticle. 
The constraints ( 12.11) can be motivated by the fact that most models of supersymmetry breaking 
predict [1] superparticles with non-vanishing color charge to be heavier than the color singlets. 
They also make it likely [TJ that sleptons can be produced in the decay of some color-charged 
superparticle, improving the chance that slepton masses can be determined experimentally. 

2.2 Event Generation 

Given the parameters of the (weak-scale) MSSM Lagrangian, we compute the supersymmetric 
spectrum with the program SOFTSUSY [5]. Next, the branching ratios of kinematically allowed 
decays are computed using SUSY-HIT [TU]. Signal events are then generated with the event 
generator Herwig++ [TT]. We first generate 10,000 events to determine the total cross section 
for the production of superparticles, and then generate the number of events corresponding to 

* This means that many of the scenarios considered in ref. [7] arc likely excluded by published analyses of 
LHC data [5] ■ We nevertheless use the same parameter sets in order to be able to directly compare our analysis 
with the results of ref. [7] . 



4 



an integrated luminosity of 10 fb _1 of data at \fs = 14 TeV. We include full QCD showering 
as well as interactions between spectator partons, using default values of the corresponding 
parameters. Since all cross sections are calculated in leading order (LO) in QCD, we use LO 
parton distribution functions CTEQ6.6 |12j . 

We do not attempt any detailed detector modeling. We do not expect experimental res- 
olutions to be very important for us, since most of the observables we employ are counting 
observables, which should be relatively insensitive to resolution effects. Acceptances (both in 
pseudorapidity and transverse momentum) are included in our definitions of observable lep- 
tons and jets, as described in Appendix A. Moreover, we assume that hadronically decaying 
r— leptons as well as b— jets can be tagged with 50 % probability within their respective ac- 
ceptance windows. We do not include false positive tags. Within the MSSM, all flavors of 
quarks and leptons are produced with comparable probabilities, so mis-tagging is not likely to 
significantly affect the distinction between different parameter sets, which is at the focus of our 
analysis. 

3 Method of Comparing Parameter Sets 

In this section we describe how we compare parameter sets. To this end we construct a total 
X 2 distribution, which allows us to compute a p— value, which in turn is used to quantify how 
similar two parameter sets appear. This is described in the last Subsection. The first Subsection 
discusses the observables we use to construct the overall \ 2 ■, an d the second Subsection describes 
the calculation of the covariance matrix for these observables. 

3.1 Observables 

If two different parameter choices yield exactly the same values for all conceivable observables, 
then there would be no chance to distinguish between these parameter sets. In principle this 
should not happen for the 15 parameter MSSM we are considering, since each parameter affects 
the mass, production cross section and/or decay branching ratios of at least one superparticle. 
However, while masses, cross sections and branching ratios formally are all observables, it is 
not clear whether they affect quantities that can actually be measured by LHC experiments 
sufficiently strongly to allow determination of the corresponding parameters from LHC data. 
Moreover, it is possible that the values of two or more parameters can be varied simultaneously 
such that no LHC observable is changed significantly. 

Chosing the right set of observables is a quite non-trivial task. On general grounds one 
expects that one needs at least one observable for each free parameter whose value one wishes 
to determineE] Supersymmetric extensions of the SM are notorious for allowing many possible 
signatures pQ; however, not all signatures are viable for all combinations of parameters. This 
argues for using not too few observables, in order to make sure that one is sensitive to all 
parameters everywhere in parameter space. 

* This is not strictly true. If some observable happens to take its absolute minimum or maximum, it alone 
would suffice to determine all free parameters. In practice this is not likely to happen for a simple observable 
(a counting rate or kinematical observable for a given final state). 



5 



Conversely, an observable should show non-trivial dependence on at least one of the free 
parameters in order to be useful. If we combine all observables into a single quantity, adding 
observables with little or no discriminatory power can dilute the effect of those observables that 
are sensitive to some parameters, reducing the statistical power of the test. 

Another argument against simply using "all" observables, as (essentially) done by Arkani- 
Hamed et al. [7], is that this makes it difficult to determine a priori the statistical correlations 
between the observables. Since our method of comparing parameter sets is based on an overall 
X 2 variable, we need the full covariance matrix between all observables, including all correla- 
tions. For example, consider two observables which are correlated and we do not take that into 
account. If we now compare two different parameter sets and the observables used for the com- 
parison are positively correlated (i.e., one tends to become smaller if the other one does, and 
vice versa), the parameter sets can look more similar than they actually are. On the contrary, 
if these observables are anticorrelated, ignoring this correlation would lead us to over-estimate 
the difference between the compared parameter sets. In fact, including an observable which is 
strongly (anti) correlated with another observable does not increase the actual amount of infor- 
mation by much, and hence does not significantly improve our chance to discriminate between 
different sets of parameters]!] Thus using more observables does not automatically improve the 
results of a parameter set comparison. 

As mentioned in the Introduction, computing the statistical correlation between a large 
number of kinematical observables constructed from the same set of events is difficult. We 
therefore use mostly counting observables, with a single kinematical observable per class of 
events. 

The production of the heavier superparticles can trigger quite lengthy decay cascades pQ, 
leading to events with several jets and/or charged leptons. If we classified all events simply by 
the number (e.g. 0, 1, 2 or 3 and more), charge and flavor of charged leptons we would already 
have 4 6 = 4096 classes of events. If we defined additional sub-classes according to the number 
of non-fe-jets (e.g. 0, 1, 2, 3, or 4 and more) and the number of 6-jets (e.g. 0, 1 or 2 and more) 
we would end up with 5 ■ 3 • 4096 = 61440 different classes. Even adding a single kinematical 
observable for each class would double the total number of observables considered. This large 
number of observables does not seem to be practical. For one thing, our event sample consists 
of typically 25,000 events after cuts, so most of the classes would be empty. 

Instead we consider 84 independent observables. The first one is the total number of events 
after cuts. Since nearly all parameter sets include a stable neutralino as LSF0, we always 
require a sizable missing Et, which greatly suppresses SM backgrounds. The details of the cuts 
depend on the number of charged leptons (meaning electrons, muons and tagged hadronically 
decaying taus), as described in Appendix [Bj On average around 30% of all supersymmetric 

t Iii the extreme case there could be a 100 % correlation between two (linear combinations of) observables. 
This would lead to a divergence of the inverse correlation matrix V -1 , because one eigenvalue of V would be 
zero. This divergence can be removed by removing one of the observables from the covariance matrix. An actual 
example of this will be described later in this Section. 

* The original parameter sets of [7] include a few examples where the LSP is a slcpton, in particular the lighter 
f eigenstate; even the set of "degenerate pairs" contains such examples. Scenarios with stable or long-lived 
charged slcpton as LSP would be trivial to distinguish from parameter sets where the LSP is a neutralino, 
using stable charged particle searches. We find that all scenarios where the LSP is not a neutralino can be 
distinguished from all other scenarios even when only using our standard observables listed below. (In all pairs 
of parameter sets we compare, at least one LSP is the lightest neutralino). 
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events pass these cuts and we only consider those events in the following. 

We divide the surviving events into twelve mutually exclusive classes depending on the 
number, charge and flavor of the measured "stable" charged leptons (l^ = e ± or fj, ±m , r— leptons 
decay within the detector and are thus non-trivial to identify experimentally). Note that 
we only include isolated electrons and muons with pseudorapidity \rj\ < 2.5 and transverse 
momentum > 10 GeV, as described in Appendix A. 1. The twelve classes of events are: 

1. 0/: Events with no charged leptons 

2. Events with exactly one charged lepton, with negative charge 

3. Events with exactly one charged lepton, with positive charge 

4. 2l~: Events with exactly two charged leptons, with total charge —2 (in units of the proton 
charge) 

5. 2l + : Events with exactly two charged leptons, with total charge +2 

6. ifl^'- Events with exactly two charged leptons, with opposite charge but the same flavor; 
i.e. e~e + or 

7. . Events with exactly two charged leptons with opposite charge and different 
flavor; i.e. e~/i + or e + fi~ 

8. Events with exactly three charged leptons with total charge —1. There is an 
opposite-charged lepton pair with same flavor. For example e _ /i~/i + or e~e~e + 

9. IfljlJ: Events with exactly three charged leptons with total charge +1. There is an 
opposite-charged lepton pair with same flavor. For example e + /i~/i + or e + e~e + 

10. l^ljlk- k^j,i for +'■ Events with exactly three charged leptons with total negative charge, 
i.e. there are at least two negatively charged leptons. There is no opposite-charged lepton 
pair with same flavor. For example e _ e~/i + or e~e~e~ 

11. I'fl'jlk. kj Lji _'• Events with exactly three charged leptons with total positive charge, 
i.e. there are at least two positively charged leptons. There is no opposite-charged lepton 
pair with same flavor. For example e + e + /i~ or e + e + e + 

12. 41: Events with four or more charged leptons 

Since we assume first and second generation sleptons to be degenerate, lepton universality 
will hold for the first and second generation also in the MSSM; there is thus little sense in trying 
to distinguish between electrons and muons. We do, however, distinguish between positively 
and negatively charged leptons. Since the initial state at the LHC is not CP self-conjugate, 
there is no reason to assume that Z + and l~ will be produced with equal rateJi] Moreover, 
pairs of charged leptons with opposite charge but the same flavor can originate from the decay 

§ In the context of the MSSM, squark decays into charginos can yield positive leptons from u and d* decay, 
and negative leptons from d and u* decays. In the absence of CP violation a sizable charge asymmetry [5] 
can only result from first generation squarks, since squarks of higher generations will always be created as 
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of a single neutralino, xl ~~ ^~X$ with j < i; all other pairs of charged leptons must come 
from the decays of two different (super)particles. In events containing a pair of oppositely 
charged leptons it therefore makes sense to distinguish pairs with equal flavor from those with 
different flavor. However, since events with four or more charged leptons are very rare, we do 
not attempt to subdivide them any further. 

For each of these twelve classes c G {1,2, ...,12} we save seven observables Oj )C , i G 
{1,2,. ..,7}: 

• Oi iC = n c /N: The number of events n c contained in the given class c divided by the total 
number of events N, i.e. the fraction of all events contained in a given class 

• O2.C = Ti CT -/n c : The number of events in a given class c that contain at least one tagged 
hadronically decaying r~ divided by the total number of events in this class 

• Oz, c = n c ^ T +/n c : The number of events in a given class c that contain at least one tagged 
hadronically decaying r + divided by the total number of events in this class 

• 04, c = n Ct b/n c : The number of events in a given class c that contain at least one tagged 
b— jet divided by the total number of events in this class 

• C = (j) c : Average number of non— b— jets of all events within a given class c 

• Oq, c = (j 2 )c- Average of the square of the number of non— b— jets of all events within a 
given class (0 

• Ojj, = (Ht)c- Average value of Ht of all events within a given class c, where Ht is the 
scalar sum of the transverse momenta of all hard objects, including the missing pt 

Jets are reconstructed using the anti— kx scheme of Fast Jet [13] . They have to satisfy 
Pt > 20 GeV and \r)\ < 4.8; b— tagging is possible only for jets with \rj\ < 2.5 that contain at 
least one (decay product of a) b— flavored hadron. r— leptons can be tagged only if they decay 
hadronically, are isolated, and their visible decay products satisfy px > 20 GeV and \r]\ < 2.5. 
The tagging efficiency for taggable r— leptons and b— jets are each 50 %, i.e. each taggable 
t— lepton or 6— jet is randomly taken to be tagged or not with 50 % probability. See Appendix 
A for further details. Since we use only a single kinematical observable for each class, and the 
classes are all mutually exclusive, we do not have to deal with correlations between different 
kinematical observables. 

Note the difference between observables 2, 3 and 4 on the one hand, and 5 and 6 on the 
other. The former three observables essentially count all events that contain at least one object 
of the given type, while the latter count the (square of the) number of jets per event. At least 
for the parameter sets considered here, the number of events containing a tagged r or b— jet 

squark -antisquark pairs. In case of the first generation, essentially only SU(2) doublet squarks can decay into 
charginos. The charge asymmetry is therefore obviously sensitive to the difference between the masses of ul and 
dh - but these masses are related by SU(2) invariance pQ, and can be taken as equal as far as LHC experiments 
are concerned. The charge asymmetry will nevertheless be sensitive to the ratio of squark and gluino masses. 
Moreover, the branching ratios of ul and dh will have slightly different dependencies on the parameters of the 
chargino sector. 

' If event i in the given class contains iV. non-6— jets, then (j 2 ) c = l/ n cEti I " 
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is usually small, and the number of events containing two or more such tagged objects is even 
smaller; it is therefore sufficient to simply count the number of events that contain at least one 
such objectjiO In contrast, most events do contain several jets. We therefore use two different 
observables to characterize the distribution in the number of jets in the different event classes. 



3.2 Covariance Matrix 

In order to perform a statistical analysis using these observables we need their full covariance 
matrix, including all correlations. This subsection contains expressions for all non-vanishing 
entries of this matrix. 

The first observable was the total number of events after cuts, N; its variance is simply 
given by 

a 2 (N) = N. (3.1) 

The next twelve observables are the fractions of events n c /N that belong to each class c. 
These twelve observables are not independent, since JZc=i n c = i- e - the fractions add up to 
unity. This difficulty could be avoided by simply using the twelve n c as variables, which are 
not correlated, and dropping the total number of events N. The reason why we include N and 
the n c /N is that we will later assign a much smaller systematical error on the event fractions 
n c /N than on the total number of events N. Note also that the event fractions n c /N are not 
correlated with the total number of events N, i.e. 

cov(^, iv)=0 (ce{l,2,...,12}). (3.2) 

On the other hand, the covariance between the fraction of events in two different classes c and 
d is nonzero: 

fn r n r '\ n r n r n r ' , , ^ , 

cov ^) = 5^ - -^f (c,c'g {1,2,..., 12}). (3.3) 

The covariance for identical classes (c = d) equals the variance, i.e. the usual error on the frac- 
tion of events in this class. The covariance matrix consisting of the twelve entries constructed 
according to eq. fl3.3p is not invertible, because the rows and columns are linearly dependent. 
This is a consequence of the constraint ^ c ^ c = N mentioned above. In fact, it should be clear 
that it is impossible to create 13 independent observables n c /N and N out of twelve measured 
event numbers n c . The solution of this problem is to exclude one n c /N with n c ^ from our 
list of observables, and hence from the covariance matrix. So at the end we are left with 84 
observables carrying different information instead of 85. 

The observables O2.C = Ti C)T -/n c , 0%^ = n CtT +/n c , 0^ c = n c ^jn c for a class c have the 
variances 

o- 2 (O hC ) = O hC ■ (1 - O hC )/n c % e {2, 3, 4}. (3.4) 

II 6— quarks will nearly always occur in quark-antiquark pairs, but the probability that both members of such 
pairs arc not only taggablc, but also tagged is rather low. Also, precisely because 6— quarks almost always 
occur in pairs distinguishing between events with one or two b— tags adds additional information only if there 
is a significant number of events with more than one bb pair in the final state; such events are very rare in our 
scenarios. 
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Observables referring to different classes c are obviously uncorrelatedE] 

Within a given class, observable 4 is obviously uncorrelated with observables 2 and 3. n T - 
and n T + have some correlation if there is a sizable fraction of events containing at least one 
t + t~ pair. In the simple situation where r— leptons can only be produced in such pairs and 
each r— lepton is identified, the number of events containing (at least) one r~ lepton would 
obviously be exactly the same as that containing (at least) one r + . In the more realistic case 
where a r has a finite (rather small) probability p T to be tagged, the correlation between n T - 
and n T + scales like p 2 , while the corresponding diagonal entries of the covariance matrix scale 
like p T . Note that p T includes the geometric acceptance and the hadronic branching ratio of 
the r in addition to the 50 % probability of tagging a taggable r. Moreover, in reality most 
t are produced together with an associated r— (anti)neutrino rather than with a second r T , 
which dilutes the correlation between observables 2 and 3 even further. We therefore ignore 
the correlation between these observables. We will check the statistical properties of our x 2 
variable later. 

The remaining observables can be written as averages over all events in a given class, 
Oi :C = (oi) c with 05 = j, oq = j 2 , 07 = Ht- Their variances can be calculated directly from the 
definition: 

a 2 (O l , c ) = ^—-((o 2 ) c -(o l ) 2 c ) i €{5,6,7}. (3.5) 
n c - 1 

Of these observables, only (j) c and (j 2 ) c are correlated within a given class: 

cov«j>„ 0' 2 )c) = -^-r • (0' 3 ) c - (j)c (f)c) ■ (3.6) 
n c - 1 

Here (j 3 ) c is also determined directly from the (simulated) events. 
3.3 Test Statistics 

Our method of comparing parameter sets is based on an overall x 2 variable, defined as: 

X 2 AB = E( - - n)V^{Oi - B n ) . (3.7) 

m,n 

Here is the prediction of parameter set A for the n— th observable and m, n run over all 
relevant observables. In general the sum in eq. fl3.7p therefore includes summation over event 
classes c and over the seven kinds of observables 0, jC listed in Subsection 13.11 We will describe 
later how we determine which observables are relevant for a given comparison; this depends on 
the parameter sets we are comparing. Finally, V~ x is the inverse of the covariance matrix V of 
all relevant observables, with entries 

V i:i = cav[Of, Of] + cov[Of , Of] . (3.8) 

* Note that we are talking about purely statistical correlations here. A fluctuation of events in one class can 
obviously not affect the number of events in a different, distinct class, nor can it affect the properties of the 
events in this other class. Different observables may very well be "physically correlated" , in that changing the 
value of some input parameter will lead to simultaneous changes in many observables, including observables in 
distinct classes. This kind of correlation need not be included when constructing the covariance matrix. It does 
complicate the estimate of errors on the parameters from an overall x 2 fit; which, however, is beyond the scope 
of this paper. 
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This corresponds to adding the errors for parameter sets A and B in quadrature. The calculation 
of the covariances for a single parameter set has been described in the previous Subsection. 

The main purpose of the present Subsection is to test the statistical properties of our x 2 
variable. In particular, we want to check that it indeed follows a proper \ 2 distribution. To 
that end we interpret eq.f l3.7p as if we were comparing two distinct measurements, each with its 
own statistical uncertainty. We use samples of simulated signal events corresponding to 10 fb _1 
of LHC data at \fs = 14TeV. We do not include any systematic uncertainties here, since 
their statistical interpretation is less clear. x\b wm follow a x 2 distribution if the difference 
between the measurements is entirely due to statistical fluctuations. We therefore compare two 
different simulations of the same parameter sets, but with different seeds of the random number 
generatorE] 

If \\b follows a proper x 2 distribution, the probability p of finding a value bigger than the 
actual one is given by 

oo 

V = J f{z,n d )dz. (3.9) 

Xab 

Here 

z (n d -2)/2 e -*/2 

/( ^ = ^ip (3 - 10) 

is the x 2 probability density function and rid is the number of degrees of freedom, i.e. the 
number of observables included in the sum (13.71) . Note that eqs. (13.91) and (13. 10[) assume that 
x\b nas been constructed from Gaussian random variables. 

As noted above, we verify the statistical properties of x\b by comparing models to them- 
selves. To that end we simulate each model twice, using different seeds in Herwig++. For 
simplicity we simulate exactly the same number of events in both runsJl] Each comparison 
yields a p— value computed from eqs. (l3.9p and (13. 10p . Performing many such comparisons leads 
to a distribution of p— values, which should be flat if x\b follows a proper x 2 distribution. This 
is a stringent test of our calculation of the covariance matrix. 

In order to achieve higher statistics we use a larger sample of parameter sets than the 384 sets 
containing the "degenerate pairs" claimed in ref. [TJ. Recall that Arkani-Hamed et. al. called 
two parameter sets indistinguishable if their quantity (ASab) 2 < 0.285. All model pairs with 
higher values of (ASab) 2 are said to be distinguishable, with the degree of distinguishability 



* Strictly speaking a \ 2 distribution is defined via the sum of the quadratic differences between Gaussian 
random variables and their means (i.e., J2 n iPn ~ O n ) ), not between the members of pairs of Gaussian random 
variables, divided by the true variance, not by its estimator. However, if both members of the pair of Gaussian 
random variables whose squared difference we are computing arc distributed with the same mean and same 
variance, a true \ 2 distribution also results for the sum over the squared differences between the two random 
variables, provided the variance is increased by a factor of two. The first condition is satisfied since both "data 
sets" are generated for the same parameter set. The second condition is approximately satisfied since we are 
normalizing by the total covariance. Of course, replacing the true covariance by its estimator obtained from 
data is standard practice in physics. 

t Using a different seed in general also leads to a slightly different estimate of the total sparticle production 
cross section. Of course, this difference decreases if more events are used to estimate the total cross section; 
recall that we estimate this cross section from simulations containing 10,000 events, leading to an uncertainty 
on the cross section from Monte Carlo statistics in the percent range. This is already much smaller than the 
physical uncertainty of the predicted cross section, e.g. due to missing higher order corrections. 
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increasing with increasing (ASab) 2 . For the present test we considered all pairs of parameter 
sets with 0.285 < (AS) 2 < 0.44, which are still relatively difficult to distinguish by the criteria 
of ref . [7] . The upper bound is chosen so that we get a few thousand additional parameter sets, 
compromising between higher statistics and limited computing power. This sample includes 
3305 parameter sets forming in 4654 pair^O in the given range of (ASab) 2 - 

We generate two statistically independent data sets for each of these 3305 parameter sets. 
For each pair of data sets corresponding to a given parameter set we separately look at the 
total number of events and at the seven kinds of observables defined for each event class; the 
event classes and observables have been described in Subsection 13.11 When analyzing the total 
number of events after cuts N obviously only a single term contributes to the definition of 
Xab m eq-( l3.7l) . In all SUSY scenarios we consider this event number is large enough to use 
Gaussian statistics. (This will be the case for all conceivable SUSY scenarios, including "no 
SUSY", after inclusion of backgrounds.) 

In contrast, the remaining observables 0^ c defined in Subsection 13. II will be approximately 
Gaussian distributed only if class c contains a certain minimal number of events. Recall that 
Xab can be expected to follow a \ 2 distribution only if the observables used in its construction 
are Gaussian variables. This is true to a good approximation only if sufficiently many events 
contribute to a given observable. 

For example, as already mentioned at the end of Subsection 13.11 the number of events 
including tagged hadronically decaying r— leptons is in general relatively small. If there is only 
one identified r event in hundred events of a given class it does not make sense to compare 
classes containing only ten events, because most of these classes would contain zero or at most 
one event with an identified r, and Gaussian statistics would not be applicable. This can be seen 
from the fact that the resulting p— value from a single comparison of these classes, computed 
according to eqs. fl3.9p and ( I3.10p . would take mostly two discrete values, as can be seen from 
the following calculation: 

V Uc > B J 



(3.11) 

Here A and B refer to the two data sets, which have been generated using the same parameter 
set, and c refers to one of our twelve event classes. The first line follows from eqs. ( 13.41) and ( 13. 71) . 
and in the second line we used the approximations n C) A ~ n C) B and -, - <C n Ct A- The 
combination t\Q = 0, = 1 or vice versa thus leads to Xab ~ 1? while n£ = nf T _ = 
obviously gives Xab = f° r this single observable. Since for small n c the observable n CjT - 
most likely takes the values or 1, the resulting distribution of p computed from this one 
observable would certainly not be flat. Instead, there would be a pronounced peak at p — 1, 

* The given (AS 1 ) 2 range actually contains 4658 pairs and 3307 parameter sets, but with two sets problems 
occurred simulating them with SUSY-HIT followed by Herwig++. 
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Table 1: The minimal numbers rij jimn are listed for our seven kinds of observables. For the 
fraction of all events that belong to a given class, n c /N, at least one of the compared data sets 
A and B has to fulfill this condition. For all other observables both data sets have to contain 
at least nj imm events in a given class c for Oj >c to be included in the calculation of Xab- 

Observable ^j, m in 

n c /N 10 

n C)T -/n c 500 

n c ,r+/ n c 500 

n c ,b/n c 50 

(j) c and(j 2 ) c 50 

(H T )c 10 



since in many pairs of data sets there would be event classes containing no identified r— lepton. 
Similar remarks apply to the other observables, although we expect more events to contribute 
non-trivially in these cases. 

In the following we therefore require a minimum number of events rij min for a given observ- 
able O iyC to be included in the computation of Xab- The values of n i>min depend on the kind 
of observable (labeled by i), as listed in Table [TJ but are independent of the event class c. We 
include 0± jC , the fraction of all events that belong to class c, in the calculation of Xab as l° n S 
as at least one of the two data sets we are comparing contains ni >m i n = 10 or more events; all 
other observables are only included if both data sets contain at least rij iimn events. The reason 
for this is that two data sets are obviously quite different, and hence distinguishable by LHC 
experiments, if one of them contains many events of a given class, while the second has none or 
very few such events. Note that the total error will then be dominated by the error on the data 
set containing many events, and should be (approximately) Gaussian. In contrast, the other 
observables will have an approximately Gaussian error only if both data sets contain sufficiently 
many events. 

The chosen minimal numbers nj >mm are shown in Tabled] They were determined by requiring 
that the distribution of p— values for a given kind of observable is at least approximately flat. 
These p— values have been obtained by collapsing the double sum in eq. fl3.7p over the index i 
labeling the type of observable to a single term. This allows us to determine the nj imm one by 
one. The choices listed in Table [1] imply that between one and twelve classes of events are used 
to determine the p— value for a given kind of observable, depending on the observable and on 
the input parameters. As expected from our previous discussion, 77, 2i min = ^3,min = 500 for the 
fraction of events in a given class containing at least one identified r~ or r + lepton, respectively, 
are the highest. For most of the parameter sets we investigate, the bulk of events are in the 
three classes with at most one lepton; as a result, these three event classes tend to contribute 
most to the calculation of x\b after the conditions summarized in Table [T] are imposed. 

We are now ready to show some results of the self-comparison of the 3305 parameter sets 
considered here. To this end, we show histograms of p— values computed from eqs. fl3.9p and 
f)3.10p ; each histogram has up to 3305 entries, since exactly two data sets are generated for each 
set of input parameters. As noted earlier, if x\b defined in eq. fl3.7p follows a x 2 distribution, 
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Figure 1: The p— value distribution of the self-comparison for the sample comprising 3305 
parameter sets. Only the total number of events after cuts, N, is included in the calculation of 

Xab and P- 

these histograms should be (approximately) flat. 

Figure [T] shows the distribution of p— values if only the total number of events after cuts N 
is used in the calculation of Xabi i- e -' n d — 1 i n eq. fl3.10p . Evidently this distribution is not 
flat. The bias towards large p— values shows that the compared pairs of data sets are more 
similar than expected if Na and N B were independent. This is due the fact that for both seeds 
of the random number generator the same number of events is simulated, which is calculated 
by multiplying the total cross section for the production of superparticles with the assumed 
integrated luminosity of 10fb _1 . In the absence of cuts we would then always have Na = Nb, 
i.e. p = 1. In practice the number of accepted events is much smaller than the number of 
generated events, and the distribution becomes quite flat for p > 0.4. Note that we fix the 
number of generated events as product of cross section and luminosity also when comparing 
different sets of input parameters. Strictly speaking, we should instead randomly select the 
number of generated events from a Poisson distribution whose mean is given by the product 
of cross section and luminosity. We do not bother to do that since this will not change the 
statistical distinguishability of two parameter sets significantly. Moreover, simply fixing the 
number of generated events is conservative, since it tends to reduce Xab- Finally, Fig. [1] shows 
that the distortion of the p— distribution is not very pronounced, except at very small p, once 
cuts have been imposed. 

The p— distributions for the fraction of events containing a reconstructed r are shown in 
Fig. |2j They follow an overall flat distribution, except for peaks at certain positions. As 
discussed above, these peaks are caused by the in general low number of events containing 
identified r— leptons. Since the LHC is a proton-proton collider and therefore has a net positive 
charge in the final state, fewer hadronically decaying r~ than r + are identified. Since we use 
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Figure 2: The p— value distribution of the self-comparison for the sample containing 3305 
parameter sets. Only the fraction of events in a given class containing an identified r~ (left) 
or t + (right) is included in the calculation of p. On average 2.8 classes satisfy the requirement 
n c > 500 (see Tabled]) and are thus included in the definition of Xab- The number of entries 
are slightly smaller than 3305 since a pair of data sets for a given set of input parameters is 
excluded if all variances (of the considered observables) in the compared classes vanish for both 
data sets or if no class fulfills the n c > 500 requirement. 



the same minimal number n2 jm i n = ^3,min = 500 of required events in a given class for both 
charges, the peaks are more pronounced for r~ events. 

For example, if only a single class contributes to the calculation of Xab 5 an d we have exactly 
one event with an identified r~ in one data set and none in the other, eq. (!3.1ip gives Xab — 1; 
the corresponding p— value for m — 1 compared observable is p ~ 0.32 which is the position 
of the first peak. Similarly, the peak at p = 1 results from pairs of data sets where each class 
contains the same number of identified r— leptons of a given charge. This yields Xab ^ a 
and hence p ~ 1. Note that pairs of data sets containing not a single identified r— lepton of a 
given charge in any event class are excluded from this comparison since in this case both the 
numerator and the denominator in our definition (13. 7\i vanish. Additionally some pairs do not 
fulfill the n c > 500 requirement for any class. As a result, the numbers of entries in the two 
frames of Figs. [2] are slightly smaller than 3305. Of course, in this case these observables cannot 
be included in the calculation of the overall x\b-> either. 

We consider the behavior of the histograms in Fig. [2] acceptable. Having too many pairs 
of data sets giving a large p— value, in particular p — 1, is again conservative. The peak 
at p = 0.32 probably underestimates the true p— value for cases where one data set has one 
identified r— lepton with a given charge and the other has zer but not by very much. Note 

§ Eq. (|3.11[) gives x\b = i n this case; the exact calculation gives a non-vanishing, but small, value, unless 
the two values of n c also happen to coincide exactly, or = = 0. 

■ The best estimate of the true expectation value of the number of events with identified r— lepton is then 
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Figure 3: The p— value distribution of the self-comparison for the sample containing 3305 pa- 
rameter sets, including all relevant observables. On average around 40 observables are compared 
for a pair of data sets. 



also that these unwarranted peaks are due to data sets containing very few identified r— leptons. 
In these cases the r observables do not contribute very much to our total Xabi an d hence do 
not affect the calculation of the overall p— value very much. 

We checked that the remaining five kinds of observables have flat distributions (not shown) 
once the conditions summarized in Table [T]have been applied. This justifies our choice of nj. m i n 
values that define when a given observable is considered relevant. 

Finally, in Figure [3] the overall p— distribution is shown using all types of observables. On 
average around 40 out of 84 observables are compared for each pair of data sets. The flat shape 
confirms our calculation method of x\bi including the calculation of the covariance matrix as 
described in Subsection l3.2l It also confirms our expectation that the anomalies we found above 
in the p— value distributions for the total number of events after cuts and for the r observables 
do not distort the overall p— value significantly 

We also performed a second kind of statistical test, using the 384 parameter sets that 
contain the indistinguishable pairs found in ref. [7]. We again performed two simulations for 
each parameter set, one corresponding to the nominal luminosity of 10fb _1 , and one with 
ten times more events. The first simulation defines the "measurements", while the second 
simulation was used to determine the true expectation values of our observables, since their 
(statistical) errors are much smaller than those of the "measurements". We then checked 
whether the normalized differences between the "measurements" and their expectation values 
follow a Student's t— distribution, as they should if the "measured" observables obey Gaussian 
statistics. These tests were also successful. However, they are somewhat less powerful, since 

1/2, again ignoring the difference in the numbers of events of the given class between the two data sets. Using 
Poisson statistics we find that the probability that our Xab > 1 is then about 0.49. 
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our computer resources did not permit to generate the required large number of events for the 
larger number of parameter sets described earlier in this Subsection. 



4 Results 

Having verified that our variable Xab behaves like a proper \ 2 distribution under appropriate 
conditions, we are now ready to use it to analyze whether pairs of parameter sets can be 
distinguished by LHC experiments. Our focus will be on pairs of parameter sets that Arkani- 
Hamed et al. [7] deemed to be indistinguishable. Recall that they found 283 such pairs formed 
from 384 different sets of parameters. 

As mentioned in the Introduction, two parameter sets A and B can be considered distin- 
guishable if a measurement can exclude the predictions made for parameter set B, under the 
assumption that Nature is described by parameter set A, or vice versa. Here the predictions 
yield the expectation values of our 84 observables, and do not have a true statistical uncer- 
tainty, given sufficiently large Monte Carlo statistics for the computation of these expectation 
values. Given our limited computer resources, we computed the predictions using ten times 
more events than generated for the measurement; this ensures that the Monte Carlo statistical 
uncertainty on the predictions is already almost negligible compared to the expected statistical 
uncertainty of the measurement, but we do take this uncertainty on the prediction into account. 

In most cases it should not make a difference if the experiment is based on parameter set 
A while the prediction is for parameter set B or vice versa. This should only be important if 
the statistical errors (for fixed integrated luminosity) of A and B differ significantly. This can 
only happen if the predictions for A and B are quite different, in which case these parameter 
sets should be distinguishable anyway. In order to have a unique comparison rule we simply 
symmetrize the expression for the covariance matrix under the exchange of A and B, i.e. we 
take the average of the covariance matrix that results when measurements based on A are 
compared to predictions based on B and the covariance matrix that describes the difference 
between measurements based on B and predictions based on A: 

v cov(O^ 10 , Gtf 10 ) + cov(Q* 10 , Og 10 ) cov(O^ 100 , Ctf 100 ) + cov(O* 100 , Og 100 ) 

Vmn - 2 + 2 

+ 5 mn (fc^ ^ 0+ 2 ^ . (4.1) 

Here the subscripts 10 and 100 refer to data sets corresponding to integrated luminosities of 
10fb _1 for the measurement and 100 fb" 1 for the predictionE] The first term in eq. fl4.ip there- 
fore describes the (expected) statistical uncertainties associated with the measurements, while 
the second term comes from the uncertainty on the predictions due to finite (although large) 
Monte Carlo statistics. Finally, the last term describes systematic uncertainties, which we as- 
sume to contribute only to the diagonal elements of the covariance matrix. We assume that 
the systematic uncertainty for a given observable O m is some fixed fraction k^ st ^ of this ob- 
servable. This is the usual ansatz at least for systematic theoretical uncertainties; experimental 



* Recall that we are predicting the expectation values of our observables. Most of these expectation values 
are independent of the integrated luminosity. The only exception is the total number of events after cuts, where 
the prediction obviously is one tenth of the number of accepted events in the larger data sample. 
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systematic errors also tend to decrease with increasing luminosity. Our numerical choices for 
the fcm yst ^ will be discussed below. 

When calculating the total Xab fr° m eq. fl3.7B . and the corresponding p— value from eqs. fl3.9p 
and (13.101) . we again only include relevant observables, defined exactly as in Subsection 13. 3[ 
For the 283 pairs found indistinguishable in ref. [7], we find that on average 32 observables 
are included in the comparison. Recall that about 40 observables were included in the self- 
comparison described in the Subsection 13.31 This is expected, since for most observables both 
data sets have to contain a minimal number of events of a given type for a certain observable to 
be included, as detailed in Table [TJ If two different parameter sets are compared it is more likely 
that one of the sets does not satisfy this criterion, thereby reducing the number of observables 
we use. 

On average about 25, 000 events pass all our cuts, for a cut efficiency of about 30 %. In 
most large majority of these events again contains no or only one isolated charged 

lepton (electron or muon), i.e. the bulk of events belongs to the first three classes described in 
Subsection 13.11 

In the following three Subsections we present results for four different assumptions regarding 
systematic errors and Standard Model backgrounds. 

4.1 No Systematic Errors, no Backgrounds 

Our first result is that in an ideal world, where the Standard Model background as well as all 
systematic errors can be neglected, all of the 283 degenerate pairs are distinguishable, i.e. lead 
to a p— value below 0.05; the mean p— value of these 283 pairs is 6.8 • 10~ 5 . Recall that ref. [7J 
also uses estimated 95 % c.l. intervals to define distinguishability. 

4.2 With Systematic Errors, no Backgrounds 

Arkani-Hamed et al. did include systematic errors. In order to allow for a direct comparison 
with their results, we also assume a relatively large systematic uncertainty of 15 % on the 
total number of events after cuts, i.e. k^ yst ^ = 0.15; this includes the luminosity error on the 
experimental side, and the uncertainty in the prediction of the total cross section on the theory 
side. Again following ref. [7] we assign much smaller systematic errors of 1 %, i.e. = 0.01, 

to all other observables. This inclusion of systematic uncertainties does not change the criteria 
defining which observables are included in the comparison. After these systematic uncertainties 
are included, 260 of the pairs deemed indistinguishable by Arkani-Hamed et al. still have 
p— values below 0.05, i.e. can be distinguished. We find that only 23 pairs have p > 0.05, which 
increases the mean p— value to 0.038. Note that most pairs have very small p— values. For 
example, only 51 pairs have p > 10~ 4 . 

In order to interpret the distribution of p— values we consider the quantity (APab) 2 intro- 
duced by Arkani-Hamed et al, which is defined by 



Here P t A and Pf are the values of the i— th parameter for parameter sets A and B, respectively, 
and n para < 15 is the number of parameters used in a given comparison. P t AB = (P A + Pf)/2 




(4.2) 
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Figure 4: The total parameter difference (AP) 2 defined in eq.f l4.2p including all 15 parameters 
versus the p— value for the 283 pairs deemed indistinguishable in ref. [7], including systematic 
errors. The p— value is shown on a linear (left) and logarithmic scale (right); in the latter case 
pairs with with p < 10~ 4 are not shown. 

is the average value of the i—th parameter for both sets. Since all our parameters are positive, 
(APab) 2 can take values in the range [0, 4]. For small values it corresponds approximately 
to the relative difference squared, i.e. (APab) 2 = 0.01 implies a 10 % difference between the 
parameters of the parameter sets A and B. 

Clearly in the limit (APab) 2 the two parameter sets A and B become identical, and 
hence indistinguishable. One could naively expect that pairs of parameter sets giving a large 
value of (APab) 2 should be relatively easy to distinguish. However, in Fig. @]we see that is not 
necessarily the case: there are pairs of parameter sets where both (APab) 2 and the p— value are 
quite large. This indicates that, at least under certain circumstances, the observables we are 
considering are not sensitive to some of our parameters. We remind the readers that all our 
parameter sets have relatively light superparticles, not much above the TeV scale, so the total 
rate of SUSY events is significant. Recall also that all pairs of parameter sets we are considering 
here were considered indistinguishable by Arkani-Hamed et al., including parameter sets with 
relatively large (APab) 2 - As noted above, our analysis shows that most of these pairs can in 
fact be distinguished. 

In the right frame of Fig. HI which uses a logarithmic scale for the p— value, the lower edge 
of the populated region includes smaller (APab) 2 for larger p— values, which conforms with 
naive expectations. However, in the left frame, which uses a linear x— axis and includes all 283 
pairs, we also find some pairs where both (Pab) 2 and p are quite small. This can e.g. happen 
if a small variation of some mass opens or closes some decay channel characterized by large 
couplings and hence potentially large branching ratios, which will significantly alter the final 
state of many SUSY events. The main conclusion therefore is that there is relatively little 
correlation between the p— value and the average parameter difference (APab) 2 computed from 
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Figure 5: As in Fig. HI except that only a single parameter has been used in the calculation of 
(AP) 2 : Mi (top left), M 2 (top right), M 3 (bottom left), \i (bottom right). 



all 15 input parameters. 

In order to gain a better understanding of this result, we consider results where only a 
subset of our 15 free parameters has been used in the calculation of (APab) 2 - We begin with 
Fig. [5J which shows results where (APab) 2 has been calculated from a single parameter in 
the gaugino-higgsino sector. We see that pairs of parameter sets that have p > 0.05 have 
gluino masses that differ by at most 7%. This is not surprising since, given our assumption 
IM3I < 1 TeV, a significant change of the gluino mass inevitably leads to a significant change 
of the total number of SUSY events. 

Among the parameters in the electroweak gaugino-higgsino sector, \x shows the smallest 
difference between pairs of parameter sets with p > 0.05, with (5hab) 2 ^ 0.15; this is to be 
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Figure 6: As in Fig. HI except that the calculation of (AP) 2 includes only the squark (left) or 
the slepton mass parameters (right). 



contrasted with (5hab) 2 < 1.5 if all pairs considered indistinguishable in ref. [7J are considered. 
The fact that /i shows the smallest difference in pairs that are difficult to distinguish can be 
understood from the observation that this parameter not only largely determines the masses 
of two neutralinos and one chargino, but also affects mixing among third generation sfermions. 
A change of \i therefore generally changes more observables than changes of the electroweak 
gaugino mass parameters M\ and Mi do. 

Most pairs of parameter sets with p > 0.05 also have quite similar values of these latter 
parameters, but there are a few outliers. This is due to the fact that our counting observables 
are mostly sensitive to the ordering of the three electroweak gaugino-higgsino mass parameters. 
For example, reducing the smallest of these parameters even more may have little impact on 
branching ratios, and may not change our sole kinematic quantity Ht very much as long as 
the lightest neutralino remains sufficiently light. Conversely, if the bino mass is larger than the 
gluino mass and also larger than the masses of the sleptons, few bino-like neutralinos will be 
produced at the LHC, so that \M\\ cannot be determined very well by LHC experiments. 

The two points with p « 0.13 and (APab) 2 > 1 in the two upper frames of Fig. [5] correspond 
to the same pair of parameters. Here the values of Mi and are (approximately) interchanged 
between the two sets. The existence of such "mirror pairs" has also been noticed in ref. [TJ. 
Note that there is a strong hierarchy between these two parameters in both cases; moreover, 
\i is quite large. As a result, basically only the lightest neutralino is produced in LHC events, 
irrespective of the ordering of Mi and Mi- In the scenario with M2 <C Mi, the lightest chargino 
is also produced frequently; however, since its mass splitting to the lightest neutralino is only 
O(100) MeV, the lighter chargino effectively behaves like the lightest neutralino in this scenario, 
as far as our observables are concerned. 

Figure E] shows results for sfermion mass parameters. In the left frame the sum in eq. (14. 2 j) 
runs over the six squark mass parameters of our model. We see that the squark masses differ 
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Figure 7: As in Fig. HJ except that (AP) 2 is calculated only from tan/3. 

relatively little within pairs of parameter sets with p > 0.05, although the spread is significantly 
larger than for the gluino mass (up to about 24 % as compared to up to about 7 %). The reason 
is that the cross section for producing a single type of squark is much smaller than the cross 
section for gluino pair production, for equal masses; this is true in particular for second and 
third generation squarks, for which no t— or u— channel processes involving two valence (u or 
d) quarks in the initial state are available. Moreover, one can again set up "mirror pairs" , e.g. 
by swapping the masses of SU (2) doublet (L— type) and singlet (R— type) squarks if all squarks 
decay directly into the lightest neutralino. 

The right frame of Fig. [6] shows corresponding results for the slepton mass parameters. Ev- 
idently parameter pairs with p > 0.05 can have quite different slepton masses. The observables 
we are using are sensitive to slepton masses essentially only if sleptons are produced in the 
decays of strongly interacting sparticles. While the conditions 02. ip make it likely that such 
decays are kinematically possible for some strongly interacting sparticle(s), they by no means 
ensure that the corresponding cross section times branching ratio is sizable. In particular, we 
saw above that bino-like neutralinos may be produced very rarely at the LHC If in addition 
the wino-like states are lighter than the sleptons, very few sleptons will be produced in the 
decays of gluinos and squarks, and our observables will not be sensitive to slepton masses. 

Finally, Fig. [7] shows that tan/3 has the largest spread among all our parameters, even when 
only pairs of parameter sets with p > 0.05 are considered. In principle this parameter has 
influence on many observables, since it appears in the chargino and neutralino mass matrices 
and also affects all sfermion masses. However, in most cases the dependence of the physical 
masses and couplings on this parameter is weak. It can be enhanced if two charginos or two 
neutralinos are close in mass, or if tan /3 is very large, in which case it modifies b and f mixing 
significantly. 
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4.3 Including Standard Model Background 



We next include Standard Model backgrounds, which had been ignored in ref. [TJ. We include 
backgrounds from Z + jets, W + jets and ti as well as single top production; the latter is 
generated using the programs MadGraph 5 [H] and Herwig++, and the other backgrounds 
are simulated directly using Herwig++. For the first two backgrounds, only leptonic W and Z 
decays are included, which have a chance to pass our cut on the missing transverse momentumJl] 
All background (and signal) contributions are only considered to leading order in perturbation 
theory. 

Altogether we find 29, 052 background events after cuts for a data sample of 10 fb^ 1 , yielding 
a signal to background ratio of very roughly 1 to 1. We add these to all our "measurement" 
samples. An independent background simulation for lOOfb -1 of data yielded 293,875 events 
after cuts, which we add to our "prediction" samples. The biggest source of background is top 
pair production, followed by the production of a W or Z boson in association with several jets. 
Of course, the background is also included in the calculation of the covariance matrix. 

Recall that a given observable is only included in the calculation of the overall Xab ^ the 
corresponding event class contains a minimal number of signal events, as listed in Table [TJ We 
now demand instead that there should be a SUSY signal of at least 3 a statistical significance 
for an observable to be considered relevant, i.e. we only include observables if the number of 
supersymmetric events in the corresponding class is greater or equal three times the square-root 
of the number of background eventsJl] In case of the event fractions in a given event class Oi jC , 
this requirement has to be satisfied for at least one of the two pairs of parameters in a given 
comparison; all other observables are included only if this new requirement is satisfied for both 
pairs of parameters. 

Including these backgrounds, but ignoring all systematic errors leads to only one parameter 
pair with p > 0.05. However, including systematic errors as well as SM backgrounds increases 
the number of pairs that cannot be distinguished "at 95 % confidence level" to 46. This means 
that our simple algorithm can resolve more than 80 % of the "degenerate pairs" found by 
Arkani-Hamed et al., even after SM backgrounds are included. 



5 Summary and Conclusions 

In this paper we investigated the question to which extent a single x 2 variable can be used to 
discriminate between different parameter sets in a general MSSM with 15 free parameters. This 
analysis was triggered by ref. [TJ, which identified 283 pairs of parameter sets that were claimed 
to be indistinguishable by LHC experiments with 10 fb _1 of data taken at = 14 TeV. In our 
analysis we used far fewer observables (up to 84, as compared to up to 1808), but took care to 
properly include all correlations, so that our Xab defined in eq.f l3.7p should behave like a proper 
X 2 distribution in the limit of Gaussian statistics; we checked this explicitly in Sec. 13.31 We saw 

t In order to reduce the number of generated events we require a minimum transverse momentum of the Z 
and W, respectively, of 100 GeV at the parton level, i.e. prior to showering. This has little influence on the 
total number of events which pass the cuts because events with lower parton-level transverse momentum have 
almost no chance to pass our missing px cut. 

* The analogous requirement on our first observable, the total number of events after cuts, is satisfied for all 
scenarios we are considering. 
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Table 2: Number of pairs of parameter sets with p > 0.05, out of the 283 pairs deemed 
indistinguishable in ref . [7J , for different levels of sophistication of our analysis (with or without 
systematic errors and Standard Model backgrounds). The mean and median values of p for all 
283 pairs are also given. 



Syst. Errors 


Backgrounds 


no. of pairs with p > 0.05 


p 


median p 


No 


No 





6.8- 10~ 5 


3.6 • 10" 146 


Yes 


No 


23 


0.038 


1.1 • 10~ 36 


No 


Yes 


1 


0.0030 


2.6 • 10~ 79 


Yes 


Yes 


46 


0.079 


1.4 • 10~ 13 



in the Introduction that this does not seem to be true for the "x 2 — like" variable considered in 
ref. [7J. We also improved the analysis by including initial state showering and the underlying 
event when simulating our signal events. We also analyzed the effects of Standard Model 
backgrounds. 

Our results are summarized in Table [2J We see that under the conditions of ref. [7J (with 
systematic errors but without backgrounds) all but 23 of the supposedly 283 indistinguishable 
pairs of parameters can in fact be distinguished at the 95 % confidence level. The median 
p— value for all 283 pairs is then of order 10~ 36 . Including also SM backgrounds increases the 
number of pairs with p > 0.05 to 46, and the median p— value increases by a factor 10 23 - but 
it still remains very small, of order 10~ 13 . We therefore conclude that our analysis strategy can 
greatly alleviate the "inverse problem" at the LHC. 

This Table also indicates that the inclusion of systematic uncertainties has a bigger impact 
than the inclusion of SM backgrounds. This is somewhat worrisome, since our ansatz for 
the systematic errors is quite ad hoc. Experimental systematic errors can only be estimated 
properly by our experimental colleagues. The rather small total systematic errors we include 
can only be realized once quantum corrections to all relevant cross sections and branching ratios 
have been computed. We are confident that this will happen if and when superparticles are 
discovered; in this we are encouraged by a very recent NNLO calculation of the cross section for 
ti production from qq annihilation [T5], which reduces the higher order uncertainty, estimated 
by varying unphysical factorization and renormalization scales, to a value below 3 %. Recall 
also that all sparticles have masses at or below 1 TeV in the parameter sets considered in ref. [7J, 
leading to quite large event samples at the LHC; this reduces the importance of backgrounds. 
As noted earlier, many of these spectra are most likely excluded by existing LHC searches; we 
nevertheless used them in our analysis to ensure that our results can be compared directly with 
those of ref. [7J . 

While we consider our results to be quite encouraging, we saw in Sec. 14.21 that in some cases 
our observables cannot discriminate between sets where at least some of the input parameters 
differ quite significantly, even if backgrounds are ignored. The pairs with p > 0.05 and the 
largest overall difference between input parameters, measured via the quantity (A?^) 2 intro- 
duced in eq. (14.21) . all have squark and gluino masses near the upper end of the scanned range, 
i.e. relatively low sparticle production cross sections and hence relatively small event samples. 
Moreover, they all have wino mass M2 <C \M±\, i.e. a light wino-like lightest chargino 
with small mass splitting to the neutralino. In fact, the mass splitting is < 200 MeV, giving 
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macroscopic decay length for the lightest chargino [IS] . A measurement of this decay length 
would give an additional constraint on our input parameters. For example, it would break the 
degeneracy between pairs that basically only differ by Mi -B- M 2 ; see the discussion of Fig. 

Similarly, we have not tried to isolate certain supersymmetric production channels. Our 
cuts have been devised solely with the purpose of suppressing SM backgrounds; no attempt has 
been made to suppress "supersymmetric backgrounds" to certain channels. For example, direct 
slepton pair production might be detectable at the LHC under certain circumstances [H]. This 
should help to resolve at least some of our degeneracies. 

Note also that our simulation does not include mixed 0(as<y\v) contributions to squark 
production cross sections, which are mostly due to interference between QCD diagrams and 
diagrams where an electroweak gaugino is exchanged in the t— or u— channel. These contri- 
butions can change some squark pair cross sections by tens of percent |18j . thereby offering 
another handle on the electroweak gaugino masses; they can also help to discriminate between 
SU (2) singlet and doublet squarks, which can be difficult if only a single neutralino state is 
accessible to squark and gluino decays. 

Finally, we have not seriously attempted to optimize the cuts, or the selection of observables, 
with a view of improve the distinguishability between pairs of parameter sets. We feel that 
this can be done in a meaningful manner only if many more than the 283 pairs of parameter 
sets we are studying are analyzed, which unfortunately is beyond the power of our computer 
resources. Besides, given that the LHC is running and producing data, the usefulness of vast 
scans of parameter space for the purpose of optimizing methods to distinguish between discrete 
sets of input parameters appears somewhat questionable. 

Instead the analysis presented here had two objectives. First, we wanted to alleviate con- 
cerns raised in ref. (7] that LHC experiments may not be able to determine many SUSY param- 
eters even if sparticles are quite light. As argued above, this goal was largely achieved. This 
indicates that our observables should also be useful for determining the values of supersymmet- 
ric parameters, rather than "only" for the purpose of discriminating between discrete sets of 
input parameters. Our second goal is therefore to motivate further studies, where (some of) our 
observables are used to determine supersymmetric parameters, as alternative, or in addition, 
to the methods that have been used so far. 
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A Definition of Observables 

In this Appendix we describe in detail how our observables are defined. In particular, charged 
leptons and jets have to fulfill certain acceptance cuts to be counted. The r— jets which arise 
from hadronically decaying r— leptons are in general just called taus here. The determination 
of b— and non— b— jets is also explained in the following. The final subsections deal with the 
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calculation of the missing transverse momentum px and the observable Ht- 

Note that we count visible particles as measurable only if they have pseudorapidity \rj\ < 5; 
neutrinos and the lightest neutralino are not considered visible, since they (usually) do not 
interact in the detector. 

A.l Electrons and Muons 

We only consider isolated electrons and muons, i.e. a charged lepton I has to fulfill the following 
criteria: 

• \r) l \ < 2.5 

• p l T > 10 GeV 

• For all measurable particles with AR = ■>/ {yf — rj) 2 + (<j) 1 — (j)) 2 < 0.2 the transverse 
energy is summed. This sum has to be Et < 5 GeV 

A. 2 Taus 

Only hadronically decaying taus, i.e. r— jets, are considered here; the electrons and muons 
produced in leptonic r decays are counted as all other charged leptons, if they satisfy the 
criteria listed in the previous Subsection. We use generator information to check whether a 
given r— lepton indeed decays hadronically. The four-momentum p T of the r— jet is then defined 
as the difference between the four-momentum of the parent r— lepton and the four-momentum 
of the corresponding v T . We impose the following acceptance cuts on the r— jets: 

• \t] t \ < 2.5 

• v t t > 20 GeV 

Furthermore the r— jet should be isolated, i.e. there must not be any charged hadrons with p? > 
1 GeV and no photons with pt > 1.5 GeV within a cone AR = y/{rf - r]) 2 + (fr - <j)) 2 < 0.5 
around the tau; note that this condition is usually more stringent than that used for electrons 
and muons. Moreover, a r— jet satisfying all conditions is only tagged with a 50 % probability. 
Recall also that some 35 % of all r— leptons decay leptonically. Altogether we thus see that a 
sample of events containing equal numbers of electrons, muons and r— leptons with identical 
kinematical distributions would indeed yield equal numbers of observed electrons and muons in 
our simulation, but far fewer identified r— leptons. Note finally that identified r— jets are not 
included in the identification of other jets. 

A. 3 Jets 

Jets are determined with the program Fast Jet [13] using the anti— k t algorithm. All measurable 
particles are taken into account, except for isolated electrons and muons that satisfy the criteria 
of Subsec. lA~T1 as well as isolated r— jets defined in Subsec. IA.21 We use the following parameter 
choices for the jet finding algorithm: 

• double dou_R = 0.5 
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• fastjet::RecombinationScheme RecSch_scheme = fastjet::E_scheme 

• fastjet:: Strategy Stra_strategy = fastjet::Best 

—> fastjet:: Jet Definition JetDef_def(JetAlgo_algo, dou_R, RecSch_scheme, Stra_strategy) 
Reconstructed jets are only counted if they satisfy: 

• p> T > 20 GeV 

• \rj j \ < 4.8 

In order to determine whether a given jet originates from a b— quark, we first check the 
progenitors of all particles in the jet (except for the photons). All particles that originate from 
the decay of one of the b— hadrons -B , B°, B + , B~ , B®, B®, A° or are marked. If a jet 
contains at least one such particle and the pseudorapidity fulfills \t] b ~^ et \ < 2.5 then the jet is 
identified as a b— jet at the generator level. Note that the number of b— jets could differ from the 
number of decaying b— hadrons already at this stage. In particular, the products of multiple 
b— hadrons could end up in one jet, which is not unlikely if the two corresponding b— quark 
momenta are nearly parallel to each other. In principle the decay products of one b— hadron 
could end up in multiple jets, but this happens only rarely. Finally, a b— jet is only tagged with 
a 50 % probability. All jets which are not tagged as b— jets are counted as non— b— jets. Note 
that we ignore the possibility that jets which do not originate from a b— quark are tagged as 
b— jets (false positive tags). 

A. 4 Missing Transverse Momentum 

In the simulation the transverse momentum vectors of all measurable particles are added. The 
missing transverse momentum p T is the negative of this sum, p T = —J2iPi,T- Our event 
selection includes a cut on the absolute value of this quantity, fx = \pr\- 

A. 5 H T 

Ht is defined as the sum of the transverse momenta of all hard objects and the absolute value 
of the missing p?. Hard objects are all isolated leptons and jets that satisfy the criteria outlined 
above. 

B Selection Cuts 

As well known, the production of heavy superparticles can be detected at the LHC on top of 
SM backgrounds only after cuts have been applied to reduce these backgrounds. We apply 
three different sets of selection cuts, depending on the number and charge of identified leptons. 
Note that we include identified r— jets as leptons for the purpose of defining these cuts. In the 
following we outline cuts for events with at most one lepton, events with exactly two opposite- 
charged leptons, and events containing at least two leptons of the same charge (and possibly 
some additional lepton(s) of arbitrary charge). All leptons and jets have to satisfy the criteria 
described in Appendix [A] 
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B.l Events with at Most One Lepton 

We impose the following cuts: 

• p T > 200 GeV 

• H T > 1000 GeV 

Additionally there are either exactly two or at least four jets, on which we apply the following 
cuts: 

B.l.l Two Jets 

This event sample will get contributions from squark pair production if at least one squark 
decays directly to the lightest neutralino (possibly plus very soft particles). 

• E 3 T > 300, 150 GeV for the hardest and second-hardest jet 

• There are no additional jets with transverse energy E 3 T > 30 GeV and no b— jets; this is 
intended to reduce backgrounds from top production 

• m" > 200 GeV 

• If there is exactly one lepton then its transverse mass with the missing transverse momen- 
tum in the event should fulfill mrp(p l , f T ) > 80 GeV; this suppresses backgrounds where 
the missing pt comes from W ± —> l^vi decays. This event sample will get contributions 
from squark pair production where exactly one squark decays into a chargino which in 
turn decays leptonically. 

B.l. 2 Four or More Jets 

This event sample will get contributions from gluino pair production, and from squark produc- 
tion in the presence of long decay chains and/or additional QCD radiation. 

• E 3 T > 100, 50, 50, 50 GeV for the four hardest jets 

• Find the smallest invariant mass of two of the four hardest jets, m^ in (there are six 
combinations). Find the smallest invariant mass of three of the four jets, m^ in (there are 
four combinations). At least one of the following three requirements has to be fulfilled, 
in order to suppress tt backgrounds: 

Either: m£ in > 100 GeV 

Or: m 3 J lin > 200 GeV 

Or: None of the four jets is a b— jet 

• If there is exactly one lepton then its transverse mass with the missing transverse momen- 
tum in the event should fulfill m,T(p l , pr) > 80 GeV; this again suppresses backgrounds 
where the missing px originates from a W decay 
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B.2 Events with Two Opposite— Charged Leptons 

We impose the following cuts: 

• p T > 100 GeV 

• If the two leptons are an e + e~ or a fi + fi~ pair their invariant mass should satisfy one 
of the following requirements, which will remove events where the leptons come from a 
Z — > decay: 

Either: m a < 75 GeV 

Or: m" > 105 GeV 

In addition, we demand that the event contains either no, at least three or at least four jets; in 
the latter two cases we apply additional cuts in order to suppress the ti background: 

B.2.1 No Jets 

• There are no jets with E 3 T > 30 GeV 

B.2. 2 Three or More Jets 

• There are at least three jets with E 3 T > 100, 100, 50 GeV 

• None of the three highest E° T jets has been tagged as a b— jet 

B.2. 3 Four or More Jets 

• There are at least four jets, with E° T > 100, 50, 50, 50 GeV 

B.3 Events with at least Two Leptons of the Same Charge 

We impose the following cuts: 

• p T > 50 GeV 

• pT > 3 • \/Ht (i n GeV). This is intended to suppress events where the missing is 
due to mismeasurements; for example it is not unlikely to find 1 TeV measured energy 
accompanied by more than 50 GeV missing transverse momentum 

• Now consider all lepton pairs with opposite charge but same flavor, i.e. e + e~, or 
t + t~. If there is no such lepton pair, the event is accepted. Otherwise we impose the 
following two additional cuts: 

If the event contains exactly three charged leptons, define the third lepton /3 as the 
one not counted in the opposite-charged same-flavored lepton pair. The event then has 
to satisfy rriT{pi 3 ,pT) > 80 GeV. If all three leptons are of the same flavor, there are two 
possible pairs, and hence two possible choices for Z 3 ; in this case this last cut has to 
be satisfied for both choices. 
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In addition, each l + l pair must be an e + e or pair with invariant mass below 

75 GeV or above 105 GeV (to suppress Z — > l + E backgrounds), or the event must contain 
at least two jets with E 3 T > 100, 100 GeV 
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